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My  Background 

•  BSME  Ohio  Northern  University 

•  SM  &  ScD  MIT/WHOI  Joint  Program 
Oceanographic  &  Mechanical 
Engineering 

•  Post-doc  at  Monterey  Bay  Aquarium 
Research  Institute 

•  Consultant  to  Disney,  BAE  Systems, 
etc.  - 

design  and  control,  robotics 

•  MIT  Research  Engineer  - 
fluid  mechanics,  biomimetics, 
underwater  vehicles 

•  MIT  Assistant  Professor  - 
marine  robots,  electric  ship, 
design  problems 


Extraordinary  Challenges  in  Marine  Systems  for 
US  Navy,  Offshore  Oil  &  Gas,  Ocean  Science,  etc. 

•  Setting: 

—  Large  physical  disturbances; 

—  Autonomy  at  all  scales  due  to  huge  domain; 

—  Dependence  on  poor  acoustic  channel; 

—  Limited  on-board  energy,  biofouling,  fouling,  traffic,  water 
pressure,  etc. 

•  Robotic  Systems:  autonomy  and  planning;  high  number  of 
agents;  integrated  mission 

•  Electric  Ship:  a  micro-grid  with  dynamic  loading,  and 
damage  scenarios 

•  MY  LONG-TERM  GOAL:  New  Design  Principles  for  Complex 
Systems  in  the  Marine  Environment 


Active  Efforts  in  My  Group 

Relaxations  and  approximations  in  DC/AC  power  system 
design;  spectral  description  of  flow  networks  (J.  Taylor) 

Ship  Hull  Inspection  Algorithms  and  Experiments 

(B.  Englot,  H.  Johanrisson,  M.  Kaess,  with  J.  Leonard) 

Design  rules  based  on  asymptotic  random  graph 
models 

Marine  Devices: 

-  vertical  glider  for  precision  seafloor  delivery, 

-  safety  valve  for  flow  control  down-hole, 

—  low-cost  acoustic  modems, 

—  quadrotors  for  HAB  outbreaks. 


ALSTOM 

Advanced 

Induction 

Motor 


Navy's  class  of  Type  45  Destroyers 


Tractor  podded  propulsors 
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QEII 


Simple  Electric  Ship  Reference  Model 
with  Complex  Dynamics 


Seven-state  nonlinear 
dynamical  system 

Fully  coupled  states 


Three-Phase  Propulsion  System 


Stiff  equations;  wide 
range  of  time 
constants 

Mechanical, 
hydrodynamic,  and 
electric  constitutive 
equations 
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An  Gap  Rotor 


Add  controllers,  user  interface,  monitoring 
s/w,  instrumentation,  etc....! 

Some  Key  Design  Challenges: 

Robustness  to  Attack/Damage, 
Reconfiguration, 
Very  Expensive  Simulations 
vs.  Scalability  of  Designs 


K.  Schmitt ,  MIT 


GPS  and  Remote  Sensing  Satellites 


Autonomous  Surface 
And  Underwater  Vehicle 
Systems 


Some  Key  Design  Challenges: 
Planning,  Integration,  Acoustics, 

Surface  Traffic  Physical  Disturbances 


Adaptive 

Sampling 


Sonars 


Advanced  Sensors 


Image:  J.  Leonard  and  H.  Schmidt,  MIT 


Offshore  Tasks  for 
Autonomous  Systems 

•  Instrument 

d  e  I  i  ve  ry/reco  ve  ry 

•  Routine  inspection 

•  Repair 

•  In-water 
decommissioning  (!) 


(Deepwater  Horizon) 
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1.  Non-complex  area 


2.  "Complex"  area 


R/V  Ocean  us  at 
WHO! 

HAUV  imaging  with 
the  Blueview 
"Microbathymetry 
Blazed  Array  "  Sonar 


H.  Johormsson,  MIT 


In-Water  Ship  Hull  Inspection  with 
Autonomous  Robots 


1.  The  Objective  and  its  Components 

The  task  forms  a  rich  and  important  robotics 
problem  that  spans  several  disciplines 

2.  Non-complex  areas:  Feature-Based  Nav 

Sonar  and  visual  imagery  both  have  a  key  role  in 
building  maps  and  navigating  with  them 

3.  Complex  areas:  Feature-Based  Planning 

Guaranteed  approximation  algorithms  to  a 
covering  tour  problem  can  provide  practical  plans 
quickly 


HAUV1B:  Built  to  work  close-in 


Two  of  eight  thrusters 


DIDSON 
pitch  axis 


DIDSON 


DIDSON:  Imaging/Profiling  Sonar 

DVL  :  Doppler  odometry  plus  four  ranges 


M.  Kokko,  MIT 


Heritage: 

Harris  and  Slate  1999:  Lamp  Ray 


Nav:  300kHz  LBL 
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Fig.  6:  Cathodic  Potential  Data  Visualization 
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Long-Baseline  Acoustic  Navigation 
flyers  and  holidays! 


Four  transponders 
and  a  moving  vehicle 
in  a  long-baseline 
configuration; 
shown  are  travel 
times,  which  encode 
distance: 
c  ~  1500m/s 


Image  from  Bahr  2009 


Ship  Inspection  Strategies  -  Open  Areas 


View 

From 

Behind 


Long  Vertical  Survey 


•  Feb.  2nd,  2006 

•  Operator  in  trailer  +  RHIB 

•  FO  tether  +  WiFi 

•  34  m  X  8  m,  2  m  spacing 

•  31  minute  long  survey 

•  DIDSON: 

-  Automatic  aiming 

-  Real-time  display 

-  Logging  both: 

•  In  the  vehicle 

•  In  the  topside 
computer 


J.  Vaganay,  Bluefin 
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Typical  Didson  Imagery 


Port  ( meters) 


Barge  Coverage 
1 — 


Image  Footprint 


Starboard 


Start/Finish 


j _ i _ 

-10  -5  0  5  10  15 

Stern  (meters) 


20  - 


10  F 


Bottom  coverage  shown 
with  DIDSON  footprints; 
dataset  first  used  for 
SLAM  (ESEIF) 


M.  Walter,  MIT 


AUVFest  2008:  Map-Building  and  Mosaicking  on  the  USS  Saratoga 


•  Nine  bucket  targets  were  planted 
on  the  hull  of  the  Saratoga  in 
rows  of  three  (the  bottom  row 
was  obscured  by  biofouling) 


Why  Ship  Hull  Inspection  is  not  necessarily  a 
"planning  under  uncertainty"  robotics  problem 

•  Structure  to  be  inspected  is  partially  known:  CAD 
models,  preliminary  scans,  human  knowledge,  etc. 

•  For  the  foreseeable  future,  humans  will  watch  and  be 
close  by 

•  Navigation  is  not  completely  dependent  on  the 
environment;  odometry  and  heading  might  be  quite 
good  over  short  time  frames 

•  100%  coverage  is  the  goal  -  does  exploration  achieve 
it? 

•  Sensor  input  is  already  difficult  enough  to  interpret! 


The  Team 


MIT  (F.  Hover,  J.  Leonard) 
Global  SLAM  (iSAM) 

Sonar  imagery  and  SLAM,  mes' 
path  planning 


Bluefin  Robotics 

(J.  Vaganay) 
Vehicle  operations, 
open-hull  lines 


University  of  Michigan 

(R.  Eustice) 

Visual  imagery  and  SI 


Florida  Atlantic 
University 

(P.-P.  Beaujean) 
Acoustic  model 


SeeByte  (S.  Reed) 
Filtering,  servos,  mesh, 
CAD/CAC 


In-Water  Ship  Hull  Inspection  with 
Autonomous  Robots 


1.  The  Objective  and  its  Components 

The  task  forms  a  rich  and  important  robotics 
problem  that  spans  several  disciplines 

2.  Non-complex  areas:  Feature-Based  Nav 

Sonar  and  visual  imagery  both  have  a  key  role  in 
building  maps  and  navigating  with  them 

3.  Complex  areas:  Feature-Based  Planning 

Guaranteed  approximation  algorithms  to  a 
covering  tour  problem  can  provide  practical  plans 
quickly 


"Cake"  Target:  Visual  vs.  Sonar  Imaging  for 
Hull-Relative  Navigation  in  Non-Complex  Area 


East  Coast  ports  RARELY 
have  good  water  clarity; 
this  is  the  best  possible 
view! 


Score:  0.1  1  1  1  1  2 

Total:  194.891281 

Count:  1754 

X:  -0.127353 

Y:  -0.192773 

T:  0.180883 

CXX*1  000:  0.000604 

CYY*  1  000  0.009969 

CTT*1  000:  0.000606 

CXY*1  000:  -0.000889 

CXT*1  000:  0.000225 

CYT*1  000:  -0.002329 

Cur  time:  1286127969.408162 

Ref  time:  1286128583.860170 

Diff  time:  -614.452008 
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Normal  Dist. 
Transform,  Biber 
&  Strosser  2003 


H.  Johannsson,  MIT 


Ship  Features  for  Hull-Relative  Navigation 


H.  Johannsson,  MIT 


Time  as  a  third  axis 


Charles  River ;  Boston 


1323  x  717  [Idle]  Ready 


H.  Johannsson  and  M.  Kaess,  MIT 


Correct  vs.  Dead-Reckoned  Path 

Charles  River,  Boston 


1349  x  759  [Idle]  Ready 


H.  Johannsson  and  M.  Kaess,  MIT 


Representative  registration  pair, 
showing  cooling  channels  and 
biofouling 


Dead-reckoned  path  over  one 
hour  vs.  feature-based  nav. 


Closing  the  Loop: 
HAUV1B  on 
King  Triton , 

East  Boston ,  MA 


H.  Johannsson  and  M.  Kaess,  MIT 


Vision  SLAM  from  Ryan  Eustice,  UMichiga 
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Point  width 


Collections 

□  Elevation  by  time 

□  Elevation  by  collection 

10.0 

Max  Elevation 


Start  ^ 
End 


Points  alpha 
□  Fill  scans 


Pose  width 


>  Camera  Sonar 
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In-Water  Ship  Hull  Inspection  with 
Autonomous  Robots 


1.  The  Objective  and  its  Components 

The  task  forms  a  rich  and  important  robotics 
problem  that  spans  several  disciplines 

2.  Non-complex  areas:  Feature-Based  Nav 

Sonar  and  visual  imagery  both  have  a  key  role  in 
building  maps  and  navigating  with  them 

3.  Complex  areas:  Feature-Based  Planning 

Guaranteed  approximation  algorithms  to  a 
covering  tour  problem  can  provide  practical  plans 
quickly 


National  Geographic 
May  2008 


Stainless  Steel 
Propeller  of  an 
Ice-Breaker:  Complex! 

Obtain  a  set  of  images 
that  covers  the 
structure,  in  minimum 
time. 

Combination  of  classic 
traveling  salesman  and 
set  cover  problems,  both 
known  to  be  NP-hard  -> 

Seek  guaranteed 
approximation  factors  in 
polynomial  time,  for 
on-site  use 


DIDSON  Profiling  Sonar  Shows  Sections  Only 


Surveying  the  propeller 
of  a  300-meter  Military 
Sealift  Command  Ship 
(propeller  about  4 
meters  in  diameter) 


Surveying  a  shaft  of  the 
same  ship  (shaft  about 
1  meter  in  diameter) 
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Oh  say  can  you  see? 

Not  your  2D  coverage  problem 


B.  Englot,  MIT 


Watertight  mesh 
on  a  7m  prop  for 
183m  USS 
Curtiss ,  from 
coarse  profiling 
sonar 

Feb  2011,  San 
Diego 


lm  props  on  a  28m  vessel 


B.  Englot,  MIT 


Watertight  mesh  on  a  21-foot  prop  for  600-foot  USS  Curtiss,  from  profiling  sonar 
Feb  2011,  San  Diego 


45  min  vehicle  run-time ,  10Hz  sampling  of  range  scans 
25k  points  subsampled  from  »lm 

B.  Englot,  MIT 


An  Outcome  of  Sampling-Based  Planning  in 
5D  Configuration  Space,  4000  Targets;  ~30%  "efficient" 

Integer  programming  solution  to  RPP  with  set  cover  constraints 
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Some  Multi-Goal  Planning  Works  &  Context 


Select  goals  in  C  to  achieve  coverage  or  reconstruct  an 

object(s),  e.g.,  Danner  &  Kavraki  2000,  Easton  &  Burdick  2005 

Given  goals  in  C,  find  feasible  path  of  minimum  cost  that 

visits  them,  e.g.,  All-Pairs  PRM  (Spitz  &  Requisha.  2000),  Lazy 
MST  (Saha  et  al.,  2006),  Ant  Colony  Opt.  (Englot  &  Hover,  2011) 

Given  targets,  covering  goals,  and  feasible  edges,  find 
min-cost  path  (VPP),  e.g.,  Scott  et  al.  2003,  Wang  et  al.  2007 


We  consider  the  whole  design  problem: 

Targets  and  obstacles  given  -  i.e.,  the  structure  only 


Multi-Goal  Path  Planning  is  Combinatorial  and 

We  Need  0(100,000)  targets  Cost  Explosion 

Approximate  the  Set  Cover  &  TSP  combined  problem  with  the 
Tour  Cover  (TC)  of  Arkin,  Halldorsson,  and  Hassin  (1993): 

Given  a  graph  with  weighted  edges,  compute  the 
minimum-cost  tour  that  is  a  vertex  cover 

Step  1:  Map  smallest  edge  weights  onto  nodes,  and  solve  the  weighted 
vertex  cover  (WVC) 

Step  2:  Condense  the  graph  around  the  edges  that  defined  the  WVC 
Step  3:  Solve  a  reduced  TSP,  and  then  expand  out  the  condensed  edges 

APXtc  £  2  APXWVC  +  APXtsp 


A  Modification  to  the  TC  Achieves  Practical  Coverage  Planning 


•  Insert  Step  0:  Use  sampling  to  generate  a  pose  cover  of 
discrete  mesh  targets;  interpret  targets  as  links  in 
configuration  space 

•  Replace  condensing  step  (2)  with  direct  edges  if  shorter 

•  Enforce  a  2-cover  bipartite  graph:  APXWVC  =  1,  in  LP  time 

•  Use  Christofides  approximation:  APXtsp£ 3/2,  in  /l//3  time 

APXtc  <  3.5  is  achievable  formally;  but  Step  0  does  not 
address  performance  of  the  cover. 


How  will  it  do? 


Some  Choices  on  the  Sampled  Cover 


Regular  Random  Entirely 

lattice  poses  on  random 

poses  manifold  poses 

< - > 

Build  cover 

Revise  and  on  the  fly;  no 

refine  cover  revisions 

< - > 


Etc. 


A  Computational 
Experiment: 

3D  cubic  domain  with  no 
obstacles 

Uniformly  distributed  point 
targets 

Vehicle  pose  [X,  Y,  Z,  hdg  ] 

Sensor  footprint  is  a  cube 
with  1%  of  domain  volume 


For  initial  graph  construction,  consider  options  (all  polynomial  time): 


A.  Set  Cover  Heuristic:  Take  first  available  cover,  keeping  all  poses  that 
see  any  new  target  (not  a  2-cover);  links  accrue. 

SC  via  rounding  LP  has  APXsc<f  (highest  multiplicity  of  sightings) 


B.  Single  Cover:  Sample  until  every  target  is  attached  to  a  pose. 
No  further  graph  work  -  each  pose  is  visited. 


C.  2-Cover  WVC:  Take  first  available  2-cover;  reject  extra  links  &  poses. 
WVC  via  rounding  LP  has  APXWVC<  2 


D.  2-Cover  Bipartite  WVC:  Take  first  available  bipartite  2-cover;  greedy 
partition  heuristic  to  maximize  targets  hit;  reject  extra  links  &  poses. 
WVC  via  LP  is  exact  APXWVC  =  1 


and  then  solve  TSP  with  Christofides 


HIGHLY 

STRUCTURED  DUMB?  BASELINE 


Computational  Experiment 

Result:  Bipartite  WVC  becomes  ~15%o  better  than  baseline  at  high  N; 

and  TOTAL  efficiency  at  100,000  targets  is  about  0.50 

Single-cover  becomes  ~5%o  better  than  baseline  at  high  N 


Sampling-Based  Coverage  Planning  for  Point  Robot  with  Cube-Sensor,  1%  of  Workspace  Volume 


Number  of  Targets  in  Inspection 


Number  of  Targets  in  Inspection 


In-Water  Ship  Hull  Inspection  with 
Autonomous  Robots 


1.  The  Objective  and  its  Components 

The  task  forms  a  rich  and  important  robotics 
problem  that  spans  several  disciplines 

2.  Non-complex  areas:  Feature-Based  Nav 

Sonar  and  visual  imagery  both  have  a  key  role  in 
building  maps  and  navigating  with  them 

3.  Complex  areas:  Feature-Based  Planning 

Guaranteed  approximation  algorithms  to  a 
covering  tour  problem  can  provide  practical  plans 
quickly 


Hard  Open  Problems  Relevant  to  the 
Marine  Inspection  Missions 

•  Better  Sensors  and  Comms 


-  .  _  " :  ’  ’  _  -  L  .  -r'  *  •'  -  -  -  *  ■ 

•  3D  SLAM  and  real-tiifie  control  on  complex 
structures 


sir. 


•  The  sealion  problem:  two  minutes 


